THE CONJUGACY SEARCH PROBLEM IN PUBLIC KEY 
■ CRYPTOGRAPHY: UNNECESSARY AND INSUFFICIENT 
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Abstract. The conjugacy search problem in a group G is the problem of recovering 
an x € G from given g 6 G and ft = x~ 1 gx. This problem is in the core of several 
Q\ ' recently suggested public key exchange protocols, most notably the one due to Anshel, 

£NJ , Anshel, and Goldfeld, and the one due to Ko, Lee at al. 

In this note, we make two observations that seem to have eluded most people's 
attention. The first observation is that solving the conjugacy search problem is not 
necessary for an adversary to get the common secret key in the Ko-Lee protocol. It is 
\^ • sufficient to solve an apparently easier problem of finding x,y £ G such that h = ygx 



for given g, h £ G. 

Another observation is that solving the conjugacy search problem is not sufficient 
for an adversary to get the common secret key in the Anshel- Anshel-Goldfeld protocol. 



> ' 1. Introduction 

One of the possible generalizations of the discrete logarithm problem to arbitrary 
groups is the so-called conjugacy search problem (CSP): given two elements g, h of a 
group G and the information that g x = h for some x £ G, find at least one particular 
t^J" ■ element x like that. Here g x stands for x~ l gx. The (alleged) computational difficulty 

of this problem in some particular groups (namely, in braid groups) has been used in 
■ several group based public key protocols, most notably in and [S]. 

In this note, we show that solving the conjugacy search problem is unnecessary for an 
adversary to get the common secret key in the Ko-Lee (or any similar) protocol, and, on 
the other hand, is insufficient to get the common secret key in the more sophisticated 
Anshel- Anshel-Goldfeld protocol. This raises the stock of the latter protocol and makes 
^ ■ one think there might be more to it than meets the eye. 



2. Why solving CSP is unnecessary 

First we recall the (generalized) Ko-Lee protocol. A group G (with efficiently solvable 
word problem) and two commuting subsets A, B C G (i.e., ab = ba for any a S A, 6 G 
B) are public. An element w S G is public, too. 

(1) Alice selects a private a £ A and sends the element a _1 wa to Bob. 

(2) Bob selects a private b 6 B and sends the element b~ 1 wb to Alice. 
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(3) Alice computes Ka = a~ l b~ l wba, and Bob computes Kb = b~ 1 a~ 1 wab. Since 
ab = ba (and therefore, a~ 1 b~ 1 = b~ 1 a~ 1 ) in G, one has Ka = Kb = K (as an 
element of G), which is now Alice's and Bob's common secret key. 

Note that since we want the key space to be as big as possible, we may assume, to 
simplify the language in what follows, that, say, the set A is maximal with the property 
that ab = ba for any a G A, b G B. 

Now suppose an adversary finds ai, 02 such that 01^02 = a wa and b\, 62 such that 
b\wb2 = b~ 1 wb. Suppose also that both 01,02 commute with any b G B. Then the 
adversary gets 

a\b\wb2a2 = a\b~ 1 wba2 = b~ l a\wa2b = b~ l a~ l wab = K. 

We emphasize that these a\, 02 and 61, 62 do not have to do anything with the private 
elements originally selected by Alice or Bob, which simplifies the search substantially. 

In other words, to get the secret key K, the adversary does not have to solve the 
conjugacy search problem, but instead, it is sufficient to solve an apparently easier 
problem which some authors (see e.g. (2]) call the decomposition problem: 

Given an element w of a group G and another element x ■ w ■ y, find any elements 
x' and y' that would belong to a given subset A C G and satisfy x' ■ w ■ y' = x ■ w ■ y. 

We note that the condition x',y' G A may not be easy to verify for some subsets 
A, but for the particular situation considered in [Hj this is straightforward and can be 
done just by inspection of the normal forms of x and y. 

The claim that the decomposition problem should be easier than the conjugacy search 
problem is intuitively clear since it is generally easier to solve an equation with two 
unknowns than a special case of the same equation with just one unknown. 

3. Why solving CSP is insufficient 

The protocol that we describe below, due to Anshel, Anshel, and Goldfeld PP, is 
more complex than the protocol in the previous section, but it is more general in the 
sense that there are no requirements on the group G other than to have efficiently 
solvable word problem. This really makes a difference and gives a big advantage to the 
protocol of P over that of [S]. 

A group G and elements a\, a^, b\, b m € G are public. 

(1) Alice picks a private x G G as a word in a±, ...,0^ (i.e., x = x(a±, ...,0^)) and 
sends bf, 6^ to Bob. 

(2) Bob picks a private y G G as a word in b±, b m and sends a\, a v k to Alice. 

(3) Alice computes x(a\, a v k ) = x y = y _1 xy, and Bob computes 

y(bf, fe^j) = y x = x _1 yx. Alice and Bob then come up with a common 
private key K = x~ 1 y~ 1 xy (called the commutator of x and y) as follows: Alice 
multiplies y~ l xy by x" 1 on the left, while Bob multiplies x~ l yx by y~ l on the 
left, and then takes the inverse of the whole thing: (y~ 1 x~ 1 yx)~ 1 = x~ 1 y~ 1 xy. 

It appears to be a common belief (see e.g. |U El El) that solving the conjugacy 
search problem for bf, b^, a\, a v k in the group G would allow an adversary to get 
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the secret key K. However, if we look at Step (3) of the protocol, we see that the 
adversary would have to know, say, x not simply as a word in the generators of the 
group G, but as a word in a±, a^. That means the adversary would also have to solve 
the membership search problem: 

Given elements x,a±, ...,ak of a group G, find an expression (if it exists) of x as a 
word in ax, a&. 

We note that the (decision) membership problem is to determine whether or not a 
given x £ G belongs to the subgroup of G generated by given oi,...,Ofc. Even this, 
apparently easier problem, turns out to be quite hard in most groups. For instance, 
the membership problem in a braid group B n is algorithmically unsolvable if n > 6 
because such a braid group contains subgroups isomorphic to Fi x F2 (that would be, 
for example, the subgroup generated by crf,a"|,cr|, and <r|, see [3]), where F2 is the 
free group of rank 2. In the group F2 x F2, the membership problem is algorithmically 
unsolvable by an old result of Mihailova [0] • 

We also note that if the adversary finds, say, some x' G G such that bf = bf , 
= &m> there is no guarantee that x' = x in G. Indeed, if x' = c b x, where Cftfe, = 6jC& 
for all i, then bf = bf for all i, and therefore b x = b x for any element b from the 
subgroup generated by b\, b m ; in particular, y x = y x ' . Now the problem is that if x' 
does not belong to the subgroup A generated by a±, ...,afe (which may very well be the 
case), then the adversary will not be able to obtain the common secret key K. On the 
other hand, if x' (and, similarly, y') does belong to the subgroup A (respectively, to the 
subgroup B generated by b±, b m ), then the adversary will be able to get the correct 
K even though his x' and y' may be different from x and y, respectively. Indeed, if 
x' = CbX, y' = c a y, where c b centralizes B and c a centralizes A, then 

x'^y'^xy = (c b x)~ 1 (c a y)~ 1 c b xc a y = x" 1 c^ 1 y' 1 c~ l c b xc a y = x~ l y~ x xy = K 

because c& commutes with y and with c a (note that c a belongs to the subgroup B, 
which follows from the assumption y' = c a y 6 B, and, similarly, q, belongs to A), and 
c a commutes with x. 

We emphasize that the adversary ends up with the corrrect key K (i.e., x'~ 1 y'~ 1 x'y' = 
x~ 1 y~ 1 xy) if and only if q, commutes with c a . The only visible way to ensure this is 
to have x' 6 A and y' £ B. 

Therefore, it appears that if the adversary chooses to solve the conjugacy search 
problem in the group G to recover x and y, he will then have to face not only the 
membership search problem, but also the (decision) membership problem, which may 
very well be algorithmically unsolvable. All this seems to be pushing the adversary 
toward trying to solve a more difficult version of the conjugacy search problem: 

Given a group G, a subgroup A < G, and two elements g,h € G, find x G A such 
that h = x~ 1 gx, given that at least one such x exists. 

Finally, we note that what we have said in this section does not affect some heuristic 
attacks on the Anshel-Anshel-Goldfeld protocol suggested by several authors [3J El Ej 
because these attacks, which use "neighbourhood search" type (in a group-theoretic 
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context also called "length based") heuristic algorithms, are targeted, by design, at 
finding a solution of a given equation (or a system of equations) as a word in given 
elements. The point that we make in this section is that even if a fast (polynomial-time) 
deterministic algorithm is found for solving the conjugacy search problem in, say, braid 
groups, this will not be sufficient to break the Anshel-Anshel-Goldfeld protocol by a 
deterministic attack. As for heuristic attacks, their limitations are explained in |10| . 
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